1. Field of the Invention
The present invention relates to a method and an apparatus for processing a plurality of data sets consisting of plural pieces of digital data and, further, it relates to a method and an apparatus for combining plural pieces of digital image data, such as image data generated by using a technique of three-dimensional computer graphics, image data (images taken from the real world) measured by using a photogrammetric technique, and image data (images taken from the real world) obtained by a technique of X-ray computer tomography.
2. Description of the Related Art
Here, in order to facilitate the understanding of problems regarding a conventional image compositing apparatus (or image compositor) and the like, a configuration and an operation of a conventional image compositing apparatus will be described with reference to FIGS. 1 and 2 which are described below in the item “Brief Description of the Drawings”.
A block diagram of an image compositing apparatus, constituted according to a conventional software technique, is illustrated in FIG. 1. The image compositing apparatus in FIG. 1 combines plural pieces of digital image data consisting of color data, opacity data and distance data in consideration of the magnitude of distance and opacity in a pixel-by-pixel manner. As shown in FIG. 1, image files 16011, 16012, 16013 and 16014, storing image data, that are to be combined with each other, a buffer 1602 in a memory, a sorting means 1603, and a compositing means 1604, are disposed in the image compositing apparatus.
By way of example, a process for combining four pieces of image data each having a resolution of 1024×1024 (1,048,576 picture elements or pixels) will be described. For the purpose of image compositing, these pieces of image data are generated, or processed, so that each of these pieces of image data is viewed from an identical viewpoint in an identical direction of the visual axis and with an identical angle of view and has an identical resolution and, further, data of the opacity and the distance from the viewpoint to an object are added to each of these pieces of image data. These pieces of image data are defined as P1, P2, P3 and P4. For example, each pixel in P1 is associated with color data (P1r, P1g and P1b), opacity data (P1a) and distance data (P1d). P2, P3 and P4 are also configured similarly.
A process for combining four pieces of image data is carried out by means of a computer program as follows.
(i) The image data P1, P2, P3 and P4 is read out from the image files 16011, 16012, 16013 and 16014, respectively, to the buffer in memory 1602.
(ii) Each of 1,048,576 pixels constituting an image plane of each of the image data is selected one-by-one. Each of the selected pixel of each of the image data is located at a position on the image plane, which is determined according to a predetermined read order, and it is defined by an intersection point between the screen and the visual axis, common to each image, when each image is viewed from an identical viewpoint in an identical direction of the visual axis and with an identical angle of view. Therefore, the images are combined with respect to each of such pixels,
(iii) Each distance data P1d, P2d, P3d and P4d of the selected P1, P2, P3 and P4, respectively, is read out from the buffer 1602 to the sorting means 1603 and the sorting means 1603 sorts the distance data in ascending order.
(iv) The compositing means 1604 reads out the color data and the opacity data in ascending order of distance according to the result of the sorting and combines images according to the following expression (1), where the color data and the opacity data are defined as C0, C1, C2, C3 and a0, a1, a2, a3, respectively, in ascending order of distance. The result of the composition that is to be obtained is defined as Cout. Here, for example, C0 is an abbreviation of (r0, g0, b0).Cout=C0×a0+C1×(1−a0)×a1+C2×(1−a0)×(1−a1)×a2+C3×(1−a0)×(1−a2)×a3  (1)
(V) The steps (iii) and (iv) are repeated for all pixels in one image plane in a predetermined order. In the case of moving video images, the steps (i) to (iv) are repeated for the number of necessary image planes.
Next, a block diagram of an image compositing apparatus, using a visibility determination technique, by a conventional depth buffer method in which it is not necessary to carry out a sorting process when opacity data is not taken into consideration, is illustrated in FIG. 2. As shown in FIG. 2 there are, image files 17011, 17012, 17013 and 17014 for storing image data that are to be combined, a buffer 1702 in a memory, a frame buffer 1703, a comparison means 1704 in a graphics LSI, and a frame buffer control means 1705 in a graphics LSI.
By way of example, a process for combining four pieces of image data each having a resolution of 1024×1024 (1,048,576 picture elements or pixels) will be described. For the purpose of image compositing, these pieces of image data are generated or processed, so that each of these pieces of image data is viewed from an identical viewpoint in an identical direction of the visual axis and with an identical angle of view and has an identical resolution and, further, data of the opacity and the distance from the viewpoint to an object is added to each of these pieces of image data. These pieces of image data are defined as P1, P2, P3 and P4. For example, each pixel in P1 is associated with color data (P1r, P1g, P1b), opacity data (P1a) and distance data (P1d). P2 P3 and P4 are also configured similarly.
A process for combining four pieces of image data is carried out, without taking opacity data into consideration, by using a graphics LSI that executes a depth buffer method at high speed as follows.
(i) The image data P1, P2 P3 and P4 is read out from the image files 17011, 17012, 17013 and 17014, respectively, to the buffer on memory 1702.
(ii) The color data and the distance data of all pixels (1,048,576 pixels) constituting one image plane of the image data P1 is written into the frame buffer 1703 controlled by the graphics LSI. Here, the image data P2 P3 and P4 can also be written into the frame buffer 1703 controlled by the graphics LSI interchangeably with P1, wherein each pixel is defined by an intersection point between the screen and the visual axis when each image is viewed from an identical viewpoint in an identical direction of the visual axis and with an identical angle of view. Therefore, the images are combined with respect to each of such pixels.
(iii) When the color data of all the pixels constituting the one image plane of the image data P2 is written into the frame buffer 1703 controlled by the graphics LSI, the comparison means 1704 compares the depth data (the distance data) of P2 with the current distance data in the frame buffer 1703 for each pixel that is to be combined. Only when the distance data of P2 is smaller than the current distance data, are the current color data and distance data in the frame buffer 1703 updated. In contrast, if the distance data of P2 is equal to or larger than the current distance data, the current color data and distance data in the frame buffer 1703 is maintained.
(iv) When the color data of all the pixels constituting the one image plane of the image data P3 is written into the frame buffer 1703 controlled by the graphics LSI, the comparison means 1704 compares the depth data (the distance data) of P3 with the current distance data written in the frame buffer 1703 for each pixel that is to be combined. Only when the distance data of P3 is smaller than the current distance data, are the current color data and distance data in the frame buffer 1703 updated and, if the distance data of P3 is equal to or larger than the current distance data, the current color data and distance data in the frame buffer 1703 is maintained.
(v) When the color data of all the pixels constituting the one image plane of the image data P4 is written into the frame buffer 1703 controlled by the graphics LSI, the comparison means 1704 compares the depth data (the distance data) of P4 with the current distance data written in the frame buffer 1703 for each pixel that is to be combined. Only when the distance data of P4 is smaller than the current distance data, are the current color data and distance data in the frame buffer 1703 updated and, if the distance data of P4 is equal to or larger than the current distance data, the current color data and distance data in the frame buffer 1703 is maintained.
According to the technique described above with reference to FIG. 1, though plural pieces of digital image data can be combined in a pixel-by-pixel manner in consideration of the magnitude of distance and opacity by reading out image data stored in the files and processing it sequentially by the computer program, numerous processes are required due to the enormous amount of data.
Further, according to a technique described above with reference to FIG. 2, that is, in a three-dimensional computer graphics LSI that has developed rapidly in recent years, though a depth buffer method or a so-called Z-buffer method is adopted to determine the visibility of polygons, this method is basically a sequential process and, when there are N pieces of image data that are to be combined, the process must be carried out N times sequentially. Further, in the method for combining images by using the depth buffer method, a sorting process is not required and each image data can be processed sequentially as described above in the steps (ii) to (v) but opacity data is not taken into consideration. Generally speaking, when three-dimensional data consisting of a group of polygons that do not intersect one another and include translucent polygons is combined, first, the visibility of all opaque polygons is determined by the depth buffer method to generate the intermediate result and, next, the translucent polygons are sorted by the distance from the viewpoint in a polygon-by-polygon manner and, then, the translucent polygons are combined in descending order of the distance from the viewpoint by carrying out an alpha blending, starting from the polygon farthest from the viewpoint, in consideration of the visibility determination and opacity. However, when image data consisting of color data, opacity data and distance data are combined, instead of the polygons, the process for sorting the image data by the distance from the viewpoint must be carried out in a pixel-by-pixel manner. Therefore, much processing time must be consumed to carry out such process.
As described above, in the above-mentioned conventional techniques, arbitrary number of image data cannot be combined in a pixel-by-pixel manner, in consideration of the magnitude of distance and opacity, in real time.